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EXECUTIVE  SUMMARY 


ISSUE 


Integrated  Air  Defense  System  (IADS)  performance  must  be  assessed  to 
determine  what  programs  need  improvements.  Exercises  such  as  those  conducted  at 
the  Joint  Combat  Identification  Evaluation  Team  (JCIET)  event  are  invaluable  for 
determining  current  capabilities  and  projecting  future  capabilities. 

BACKGROUND 

The  JCIET  program  is  well  established  and  serves  many  needs  of  the  Services. 
JCIET’s  primary  mission  is  to  evaluate,  investigate,  and  assess  Joint  integration  and 
interoperability  of  systems,  concepts,  capabilities,  TTP,  and  doctrine  directly  affecting 
Combat  Identification  (CID)  within  the  present  and  future  Joint  battlespace.  The  JCIET 
02  event  offers  the  Single  Integrated  Air  Picture  System  Engineering  Task  Force  (SIAP 
SE  TF)  the  opportunity  to  gather  empirical  data  to  support  analysis  efforts  such  as, 
calibration  of  modeling/simulation  and  Hardware-ln-the-Loop  (HWIL)  tools,  and 
perturbation  analysis  studies. 

APPROACH 

This  document  is  an  appendix  to  the  Standard  SIAP  Data  Management  and 
Analysis  Plan  (DMAP).  This  event  -specific  appendix  includes  a  high-level  description 
of  the  JCIET  02  event,  roles  and  responsibilities  of  on-site  teams  and  an  overall 
approach  as  to  how  the  analysis  process  will  be  conducted  for  this  event. 

SCOPE 

This  Appendix  is  intended  to  be  an  end-to-end  document  for  the  JCIET  02  event 
for  the  SIAP  SE  TF  that  ranges  from  planning,  through  analysis  and  result  reporting,  to 
data  storage  and  lessons  learned. 
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1.  INTRODUCTION 


The  purpose  of  this  appendix  is  to  provide  guidance  for  the  collection  of  data 
during  the  JCIET  02  event  and  the  analysis  of  that  data  in  support  of  SIAP  SE  TF 
efforts.  The  SIAP  Analysis  Team  (SAT)  is  responsible  for  the  planning  and  execution  of 
this  plan.  The  goal  of  this  document  is  to  convey  the  following: 

1 .  Provide  a  description  of  the  applicable  experiments  to  evaluate  SIAP  systems 
that  will  be  conducted  during  JCIET  02 

2.  Explain  how  the  SAT  will  use  the  data  collected  at  events  to  assess  Joint 
Integrated  Air  Defense  Systems  (IADS)  performance 

3.  Provide  a  list  of  the  data  required  to  support  the  analysis  efforts  described  in  2, 
above 

4.  Provide  a  schedule  and  outline  roles  and  responsibilities  of  participants  (SAT, 
Services,  test  staff,  etc)  before,  during,  and  after  the  event 


1.1  Background 

JCIET  is  a  US  Joint  Forces  Command  organization  chartered  to  discover  and 
investigate  Combat  Identification  shortfalls  and  recommend  solutions.  The  purpose  of 
JCIET  02  is  to  employ  current  and  leading-edge  service  systems,  capabilities  and 
concepts  in  an  operationally  realistic  joint/combined  combat  scenario  to  assess  tactics, 
techniques  and  procedures  (TTP),  systems  and  architectural  advances.  During  its  10- 
year  history,  the  JCIET  evaluations  have  been  the  catalyst  for  fixing  many 
joint/combined  TTP  and  system  interoperability  issues  affecting  the  services,  across 
mission  areas  (surface-to-surface,  air-to-surface,  surface-to-air,  and  air-to-air).  The 
JCIET  02  event  offers  the  Single  Integrated  Air  Picture  System  Engineering  Task  Force 
(SIAP  SE  TF)  the  opportunity  to  gather  empirical  data  to  support  analysis  efforts  such 
as,  calibration  of  modeling/simulation  and  Hardware-ln-the-Loop  (HWIL)  tools,  and 
perturbation  analysis  studies. 


1 .2  Overview  of  SAT  Analysis  Efforts 

This  section  provides  a  brief  overview  of  SAT  Core  Working  Group  members, 
goals  and  objectives,  expected  products,  and  a  schedule  for  JCIET  02-related  analysis 
efforts. 
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1 .2.1  SIAP  Analysis  Team  (SAT) 

The  SAT  Core  Working  Group,  listed  in  Table  1,  will  be  responsible  for  the 
overall  planning  and  execution  of  SIAP  SE  JCIET  02  analysis  efforts.  The  SAT  Core 
Working  Group  for  JCIET  02  will  be  comprised  of  subject  matter  experts  representing 
the  systems  that  participated  in  the  event,  and  functional  area  subject  matter  experts 
that  provide  expertise  in  a  specific  technical  area. 


Table  1.  SAT  Core  Working  Group  (JCIET  02) 


Member 


Darrell  Schultz 


ARMY 


Patrick  Duggan 


Danny 

Ellenburg 


John  Jordan 


NAVY 


Mary  Rock 


Function 


Project 
Officer/Lead 
Systems  Engineer 


Lower  Tier 
Program  Office 


PATRIOTSystem 
and  Functional 
SME 


PATRIOT  System 
and  Functional 
SME 


Phone  e-mail  _ 


(703)602-6441  ISchultzdo@navsea.navv.mil  I 


(256)  955-3867  Patrick.duggan@lowertier.red 
stone.army.mil  _ 


(256)895-2481  Daellenb@ingr.com 


(256)  895-7684  John.iordan@cas-inc.com 


Erik  Van  Fleet 


MARINE 

CORPS 


Jim  Green 


AIR  FORCE 


Felix  Noboa 


Diane  Griffin 


Bert  Pryor 

Others 

Wayne 

Altrichter 


Jim.qreen  @  rnbtechnoloqies.c 

om 

E-2C  System  and  (301)757-1576  lrockmf@navair.naw.mil  I 
functional  SME  _ 


AEGIS  System  (909)273-4155  IVanfleetet@corona.naw.mil 
and  Functional 
SME 


TAOC  System 
SME 


AW  ACS  System 

SME _ 

AW  ACS  System 
SME 


AW  ACS  System 
SME 


Functional  Area 
SM  E/Analyst 


(540)  659  -3505  I  om 


(781)377-7155  Fnoboa@mitre.or 


(405)  734-  2734  Diane.qriffin  @tinker.af.mil 

(405)  734-3073  Bertram.orvor@tinker.af.mil 


(973)305-2120  Altrichter®  baesvstems.com 
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Ted  Rice 

Functional  Area 
SME 

(315)  456-3377 

Theodore.r.rice@lmco.com 

Paul  Symborski 

Functional  Area 

SM  E/Analyst 

(703)  824-2424 

Svmborso@cna.orq 

Dr.  Larry  Lewis 
Bruce  Behrens 

Functional  Area 

SM  E/Analyst 

(850)  882-6700 

Lawrence.lewis@ealin.af.mil 

Bruce.behrens@eqlin.af.mil 

1.3  Goals 


The  SAT  goals  for  JCIET  02  are: 

1 .  Evaluate  IADS  performance  against  Theater  Air  and  Missile  Defense  (TAMD) 
and  CID  Capstone  Requirements  Documents  (CRDs)  within  the  architecture 
present,  and  identify  shortfalls 

2.  Identify  root  cause  of  these  shortfalls 

3.  Evaluate  candidate  solutions  for  improving  IADS  performance 

4.  Support  verification  and  validation  of  proposed  solutions 

5.  Provide  recommendations  for  improving  test  venues  to  better  support  IADS 
performance  assessment  efforts. 

1 .4  Objectives 

The  specific  objectives  for  this  JCIET  02  event  are  defined  by  critical  experiments 
that  are  described  in  detail  in  Chapter  3  of  the  Standard  DMAP. 


1.5  Products 

The  expected  products  of  the  SAT  efforts  for  JCIET  02  are  the  following: 

1 .  Evaluation  of  SIAP  performance  based  on  SIAP  attributes  and 
implementation  methodologies 

2.  Identification,  documentation,  and  resolution  of  issues  and  deficiencies 
related  to  the  event 

3.  Data  for  model/tool  verification  and  validation 

4.  Identification  and  documentation  of  capabilities,  limitations,  and  lessons 
learned  for  tactical  operations 

5.  Refinement  of  standardized  processes,  tools,  and  collaborative  analysis 
methods 

1.6  Schedule 

Figure  1  provides  a  schedule  of  SAT  activities  for  the  JCIET  02  event. 
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ID 

Task  Name 

1 

JCIET  2002 

2 

DMAP  Appendix  (Draft) 

3 

DMAP  Appendix  (Final) 

4 

Test  Readiness  Review 

5 

Spin  Up  Week 

6 

Event  Data  Collection 

7 

Analysis 

8 

Data  Availability  Matrix 

9 

Run  Selection 

10 

Track  Matching 

11 

SIAP  Metrics 

12 

Root-cause 

13 

Analysis  Meeting 

14 

Analysis  Meeting 

15 

Analysis  Meeting 

16 

Analysis  Meeting 

17 

Analysis  Meeting 

18 

Analysis  Report  (Draft) 

19 

Analysis  Report  Review 

20 

Analysis  Report  Final 

Apr  |  May  [  Jun  |  Jul  |  Aug  |  Sep  j  Oct  |  Nov 


Figure  1 .  JCIET  02  activity  schedule 

1.6.1  Daily  Schedule 

Figure  2  provides  the  schedule  for  daily  JCIET  activities. 


Week  1  0700  0800 
Week  2  1500  i6oo 


-o  Element 
o  brief 
“  STTO 


Vulnerability  Period 


◄—  De-brief  Process  — ► 

Phase  1  Phase  2  Phase  3 

..  . 

Crew  Element  Mission  ^  D 


SAT  On-site 
Analysis  @ 
CRTC  Bid  5 


Mission  planning  for  next  day 

SAT  On-site  Analysis  @ 
CRTC  Bid  5 


Figure  2.  Daily  schedule  of  JCIET  activities,  15-25  April  02. 

1.7  Event  Planning 

Table  2  provides  an  event  planning  worksheet  for  the  JCIET  02  event. 
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Table  2.  JCIET  02  Event  Planning  Worksheet 


Name  of  Event: 

Date(s): 

Location: 

Type  (HWIL,  Live): 

JCIET  2002 

15-26  Apr  2002 

Gulfport,  MS 

Camp  Shelby,  MS 

Live 

Systems  participating: 


QTY  System  Name/Service 

1  PATRIOT/ARMY 

2  AEGIS/NAVY 
E2-C/NAVY 
TAOC/USMC 

E-3  AWACS/USAF 
RC-1 35S  Rivet  Joint/USAF 


List  critical  experiments  to  be  conducted  (using  standard  DMAP  Chapter  3  designations).  Include  any 
discrepancies  between  how  experiment  will  be  conducted  and  the  experiment  description: 

Critical  experiments  1-7,  9,  10 


Data  Extraction:  Provide  description  of  discrepancies  between  data  extraction  nodes  to  be  used  and  those 
described  in  Section  4.2.1. 


Model/Tool  Description  for  processing  and  analyzing  data  (i.e.  MSI,  built-in  analysis  tools): 
JCIET  processes  and  conducts  its  own  analysis  of  the  JCIET  event. 
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Software  Version  Data  collection  type 

(if  applicable)  (Automated,  manual,  both) 


2.  PROCESS  FOR  iADS  PERFORMANCE  ASSESSMEN I 
2.1  Critical  Experiments 

It  is  anticipated  that  the  participants  and  activities  planned  to  occur  during  JCIET 
will  satisfy  the  SIAP  critical  experiments  listed  below.  Following  the  JCIET  event  the 
SAT  will  determine  which  vignettes  provided  sufficient  participation  and  data  to  support 
a  specific  critical  experiment. 

1 .  Time  Synchronization  -  System  synchronization  and  latency  deficiencies  are 
endemic  to  failure  to  a  achieve  SIAP  and  result  in  warfighting  capability  shortfalls. 

2.  Sensor  Trackina/Reportina  Accuracy  -  Poor  tracking  performance,  compounded 
by  inaccurate  TQ  reporting  and  other  MIL-STD-6016A-related  deficiencies 
contribute  to  warfighting  capability  shortfalls. 

3.  Data  Registration  -  Uncompensated  data  registration  errors  have  a  significant 
adverse  impact  on  the  SIAP.  Systems  must  implement  standardized  sensor 
registration,  aligning  sensors,  and  inertial  navigation  systems  to  a  common 
geodetic  reference. 

* 

4.  Automatic  Local-to-Remote  Track  Correlation/Decorrelation  -  Not  all  systems 
implement  automatic  correlation/decorrelation  processing,  and  differences  in  the 
methods  employed  degrade  warfighting  capability. 

5.  Identification  Processing  -  Mis-identification  is  frequently  caused  by  erroneous 
IFF/SIF-to-track  association.  Additionally,  differences  in  the  way  automated 
identification  and  data  fusion  systems  behave  adds  to  failure  to  achieve 
commonality. 

6.  Formation  Tracking  and  Assessment  -  Can  systems  automatically  assign  the 
identification  of  a  track  that  has  been  declared  to  have  a  strength  greater  than 
one  to  the  tracks  within  the  prescribed  formation  "window”,  How  often  will  this 
processing  provide  the  correct  identification?  The  incorrect  capability? 

7.  Model  and  Simulation/Stimulation  Fidelity  -  Combination  of  modeling/simulation, 
HWIL  and  open-air  events  provide  sufficient  fidelity  to  reflect  operational  system 
performance  to  support  assessment  and  engineering  efforts  to  predict  warfighting 
improvements  of  implementing  ICPs. 

8.  PPLI  Accuracy  -  PPLIs  are  believed  to  be  very  accurate;  however,  due  to  data 
link  latencies  and  the  fact  that  some  navigation  systems  are  not  integrated  with 
the  data  link,  inaccuracies  may  be  present.  These  inaccuracies  will,  if  present, 
degrade  the  network  navigation  solution  and  data  registration  performance. 
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9.  Multi-Link  Translation/Forwardinq  -  There  is  a  need  to  reliably  translate  and 
forward  information  from  one  tactical  data  link  to  another  tactical  data  link  in  ways 
that  support  the  SIAP. 


2.2  Operational  Context 

Since  JCIET  is  a  live-fly  exercise  the  participant  forces  operate  in  a  manner 
similar  to  that  of  a  real  conflict.  As  such,  the  scenarios  and  flight  profiles  are  dynamic, 
unscripted  events.  In  order  to  relate  JCIET  02  events  to  the  SIAP  Common  Reference 
Scenario  (CRS)  the  SAT  will  have  to  review  selected  vignettes  and  determine  the 
commonality  to  the  CRS. 

2.3  IADS  Performance  Assessment 

The  assessment  of  the  IADS  performance  will  be  conducted  by  calculating  the 
SIAP  metrics,  (attributes  and  associated  Measures  of  Performance  (MOPs)),  performing 
root-cause  analysis  and  perturbation  analysis  studies.  The  majority  of  the  performance 
assessment  analysis  will  be  conducted  post-event,  however  significant  root-cause 
analysis  will  be  conducted  on-site. 

2.3.1  SIAP  Metrics 

The  SIAP  metrics  analysis  will  be  conducted  in  accordance  with  Sections  3.3  and 

3.4  of  the  Standard  DMAP.  This  analysis  will  take  place  post-event. 


2.3.2  Root-Cause  Analysis 

Root-cause  analysis  in  support  of  the  IADS  performance  assessment  will  be 
conducted  both  on-site  during  JCIET  02  and  post-event.  During  JCIET  02  the  SAT  will 
be  located  in  Building  5  of  the  Air  National  Guard  Combat  Readiness  Training  Center 
(ANG  CRTC)  located  in  Gulfport ,  MS.  The  layout  of  the  SAT  in  Building  5  is  provided  in 
Figure  3.  Each  day  of  the  JCIET  02  event  the  SAT  will  select  specific  Events  Of  Interest 
(EOI)  discussed  during  the  JCIET-run  Air  Defense  mass  debrief  for  further  analysis. 
These  events  are  normally  associated  with  the  loss  of  Blue  Force  assets  either  through 
fratricide  or  a  leaker  situation.  The  SAT  will  reconstruct  these  selected  EOls  using  the 
Warfare  Assessment  Model  (WAM)  and  system-level  analysis  tools  to  determine  the 
root-cause  of  the  event  in  sufficient  detail  so  as  to  recommend  changes  to  system  set¬ 
up  or  operator  actions  for  the  remainder  of  the  event; 
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Figure  3.  Building  5  Layout 


2.3.3  Test  Observation  Reports 

During  the  SAT’s  review  of  the  EOls,  or  during  the  daily  vulnerability  periods, 

SAT  members  may  generate  Test  Observation  Reports  (TORs).  These  TORs 
document  a  perceived  anomaly,  point  of  interest,  incident  or  situation  that  requires 
further  analysis.  The  SAT  will  collect,  review  and' attempt  to  resolve  any  TORs  during 
the  on-site  analysis  effort.  Any  TOR  that  is  unresolved  at  the  end  of  the  on-site  analysis 
effort  will  be  considered  for  further  analysis  during  the  post-event  analysis  effort. 

2.4  Perturbation  Analysis 

The  SAT  will  conduct  perturbation  analysis  using  JCIET  02  data.  The 
ODDSCAPE  tool  used  by  the  SAT  and  discussed  in  the  standard  DMAP  is  designed  to 
support  perturbation  analysis  for  the  JCIET  02  event. 

2.5  Post-Event  Analysis  Efforts 

Following  JCIET  02  the  SAT  will  meet  in  conjunction  with  the  JCIET  analysts  at  the 
JCIET  headquarters,  Eglin  AFB,  on  a  periodic  basis  to  complete  the  IADS  performance 
assessment  analysis  (SIAP  Metrics  and  root-cause).  Based  on  previous  post- 
ASCIET/JCIET  analysis  efforts  it  is  planned  that  the  SAT  would  hold  five,  week  long, 
analysis  meetings.  The  first  of  these  would  be  the  week  of  13  May  and  occur  every  5 
weeks  until  30  Sep  02,  as  indicated  in  Figure  1 . 
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3.  EVENT  EXECUTION  AND  DATA  REQUIREMENTS 

3.1  Test  Plan  and  Supporting  Documents 

JCIET  has  developed  and  will  be  providing  the  overall  test  plan  for  this  event. 
Section  3.1 .1  gives  an  overview  of  the  JCIET  event. 

3.1.1  JCIET  Scenario  and  Participating  Systems 

Figure  4  shows  a  map  of  the  JCIET  scenario  for  the  2002  event.  The  mission 
areas  and  Services  participating  are  indicated  to  the  left. 


•  Air  defense/ 
air-to-surface 
mission  areas 

•  Supports  objectives 

-  USAF  El 

-  USN  0 

-  USA  0 

- USMC  0 

-  SIAP  SE  0 

- JCMD  0 

-  JC2ISR  0 

-  U.K.  0 

CP-01 -05  (from  Map01-08) 

Figure  4.  JCIET  02  scenario 

3.1.1 .1  Participating  Systems:  Army 

Table  3  shows  the  participating  Systems  for  the  U.S.  Army. 


Hagler  j 


-180  NM- 
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Table  3.  Army  participating  Systems 


'  :  ;  .  BLUF^R0-:g:;«  f;  ^ 

.  '  Q p FOR  1  p 

IIKSlli  Platform  . ;  1  1 

Org. 

imm 

0 

Patriot 

1-1  ADA 

FAAD  Avenger 

4-200  ADA  BN 

FAAD  sensor/C2  platoon 

1/3  ADA 

ABMOC 

GRCS-2 

15  Ml  BN 

Hunter  UAV 

Tact.  UAV  Office 

SOF 

ODA  774 

3.1. 1.2  Participating  Systems:  Navy 

Table  4  shows  the  participating  Systems  for  the  U.S.  Navy. 


Table  4.  U.S.  Navy  participating  Systems 


Hi,  lljll  BLUFOR  IHIp ,,  j  i  '  ,  ■ 

Platform 

Platform 

USS  Cape  St.  George 

F/A-18 

NAWC 

USS  Anzio 

F-14D 

VX-9 

JMAST  (TFCC) 

CINCLANTFLT 

EA-6B 

VAQ-131 

E-2C 

Naval  Air  Warfare  Center 

P-3 

NAS  Jacksonville 

NP-3 

CFMCC 

CCDG-8 

3.1 .1.3  Participating  Systems:  USMC 

Table  5  shows  the  participating  Systems  for  the  U.S.  Marine  Corps. 
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Table  5. 


USMC  participating  Systems 


BLUFOR 

OPFOR  :  ihi-;  .&  * r 

V':' 

i  ■  is  imtiii  iiiii  iiiii  mu 

i ;  .  ■;  ■■  I"  .  Org.  ly 

HI  j  Org.  -  1'IHI 

UH-1N 

HM LA-773  DCTA 

EW/C 

MACS  2 

AH-1W 

Tank  Co.  (-) 

C  Co.,  8th  Tank  BN 

AV-8B 

VMA  542 

INF  PLT  (-) 

8th  Comm  BN 

AO/TACP 

3d  ANGLICO 

TAOC  (-) 

MACS  24 

Avenger 

4  LAAD  BN 

Tank  Co.  (-) 

1st  Tank  BN 

INF  PLT 

3d  BN,  23d  Marines 

3.1. 1.4  Participating  Systems:  Air  Force 

Table  6  shows  the  participating  Systems  for  the  U.S.  Air  Force. 

Table  6.  U.S.  Air  Force  participating  Systems 


IJIIJ  BLUFOR  ■  1 1  V  I  m 

Platform 

;;  Org.  -- 

ggpiatform':' 

HHESHH ! 

CRC 

255  ACS 

B-1 B  (3) 

28  OSS 

EC-130E  ABCCC 

42  ACCS 

QF-4  (2) 

82  ATRS 

F-15C 

3  WG/90  FS 

CRE 

SEADS 

RC-135S  Rivet  Joint 

55  WG/97  IS 

E-3  AWACS 

3  WG/962  AACS 

E-3  AWACS 

964  AACS 

E-8C  JSTARS 

93  ACW 

F-15E 

422  TES 

40  FTS 

TBMCS 

C2TIG 

3.2  Test  Item  Description  and  Configuration 

During  the  spin-up  portion  (a  week  prior  to  the  JCIET  event),  it  is  imperative  that 
each  system  representative  ensures  that  the  data  extraction  points  listed  in  the  data 
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collection  matrices  are  working  and  providing  the  data  expected  for  conducting  the 
critical  experiments  (e.g.,  at  the  appropriate  data  rates,  are  recording  properly). 


3.2.1  Data  Extraction  Diagrams 

Participants  will  follow  the  data  extraction  diagrams  provided  in  the  standard 
DMAP. 

3.2.2  Data  Collection  Requirements 

Each  system  representative  is  responsible  for  collecting  data  that  will  be  read 
into  the  PET  tool.  The  variables  and  formats  needed  are  provided  in  the  standard 
DMAP. 

3.3  Data  Management  and  Storage 

For  purposes  of  supporting  root-cause  analysis  efforts,  the  participants  are 
required  to  maintain  a  record  of  their  data  for  a  period  of  five  (5)  years.  Participants  are 
requested  to  provide  the  SAT  with  contact  information  for  accessibility  of  data  as  shown 
in  Table  7. 


Table  7.  Data  Management  and  Storage  Worksheet 


Name  of  Person/Organization  providing  information 

Rudy  Alaniz/  NSWC  Corona 

System  Identification  (incl.  s/w  version) 

USS  CAPE  ST  GEORGE  (CG  71 ) 

Description  of  data  collected  and  available,  including  POC  and  accessibility  information 

CEC  data,  AEGIS  data  including  SPY,  C&D,  WCS,  SGS  WAM  file,  and  WCS  shot  log.  POC,  Erik  Van  Fleet 

Name  of  Person/Organization  providing  information 

Rudy  Alaniz/  NSWC  Corona 

System  Identification  (incl.  s/w  version) 

USS  ANZIO  (CG  68) 

Description  of  data  collected  and  available,  including  POC  and  accessibility  information 

CEC  data,  AEGIS  data  including  SPY,  C&D,  WCS,  SGS  WAM  file,  and  WCS  shot  log.  POC,  Erik  Van  Fleet 

Name  of  Person/Organization  providing  information 

Tom  Potochny/NSWC  Corona 

System  Identification  (incl.  s/w  version) 

CEC 

Description  of  data  collected  and  available,  including  POC  and  accessibility  information 

Receive  CEC  data  from  Applied  Physics  Lab  personnel  from  the  TPS-59  and  E-2C  (2000). 
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3.4  Limitations 


There  are  no  limitations  imposed  on  the  participants  for  the  JCIET  02  event. 


4.  DATA  ANALYSIS  PROCESS 

The  SAT  is  responsible  for  all  data  analysis  and  reporting  efforts  associated  with 
the  JCIET  02  event.  The  SAT  will  perform  an  IADS  performance  assessment,  prepare 
individual-performance  reports  for  respective  participants’  systems,  and  prepare  group- 
performance  assessments  for  Government-only  use. 

4.1  Data  Availability  Matrix 

The  SAT  members  will  be  responsible  for  reviewing  the  data  recorded  by  their 
systems  and  report  on  what  data  is  available.  The  SAT  will  collect  all  of  the  systems’ 
reports  and  identify  the  time  sets  to  be  analyzed. 

4.1.1  Critical  Experiments 

The  SAT  will  review  the  data  availability  matrix  that  is  developed  and  identify 
events  where  sufficient  data  to  support  the  critical  experiments  exists.  The  tables  in  this 
section  provide  a  cross-reference  of  the  critical  experiments  to  the  system  level  data 
collection  capability/points. 


Table  8.  Time  Synchronization 


Time  Synchronization  and  Navigation 

PATRIOT 

AEGIS 

E-2C 

TAOM  V4 

AWACS 

1 

Time  tagged  platform  navigation  data 
(see  Note  1)  from  platforms  that  have 
their  system  time  slaved  to  GPS.  It  is 
necessary  to  have  systems  that  provide 
measurements  at  the  same  time  the 
(GPS)  truth  pods  supply  their 
measurements.  In  this  way,  it  is  possible 
to  determine  what  measurements  were 
taken  at  a  given  instant  of  time  by 
comparison  of  their  values,  i.e., 
comparison  of  truth  pod  GPS  position 
data*  @  1  Hz  with  host  navigation  GPS 
position*  @  1  Hz.  Then  the  two  times  can 
be  compared  to  determine  accuracy  (i.e., 
measurements  are  used  to  correlate  the 
time),  (see  Note  2) 

ECS 

EDR 
provides 
GPS 
position 
with  time 
tags  (3) 

C&D  War 
Diary, 

EPN 

43,92,19 

5,198. 

C2P 

batch 

sets, 

SGS 

TPS-59 

SGF  Files 

Yes,  at 
the 

terminal 
on  the 

NCS  bus 

! 
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2 

Time  and  WGS-84  position  data*  from 
GPS  truth  pods  mounted  on  these 
platforms.  Most  of  the  platforms  that  fall 
in  this  category  are  not  slated,  at  this 
time,  to  be  equipped  with  truth  pods  - 
AEGIS  ships,  E3  (AW ACS),  F-15,  E2-C, 
etc...? 

N/A 

N/A 

N/A 

Same  as 
above 

Note  1 :  Navigation  data  consists  of  the 
typical  inertial  navigation  state  vector 
data,  i.e.,  WGS-84  position,  velocity, 
attitude,  and  attitude  rate.  This  data 
should  be  recorded  at  a  minimum  rate  of 

8  Hz. 

Note  2:  Stationary  ground  units  are  not 
expected  to  have  recorded  navigation 
state  vector  data  as  described  in  Note  1 . 
However,  a  GPS  surveyed  location  of  the 
platform  sensor  and  Link  16  transmit 
antenna  should  be  provided. 

*  GPS  Position  and  WGS-84  Position  is 
with  respect  to  Mean-Sea-Level  (MSL) 
Altitude. 

Table  9.  Sensor  T rack-Report  Accuracy 


— 

Sensor-Track  Report  Accuracy 

PATRIOT 

AEGIS 

E-2C 

TAOM  V4 

AWACS 

r 

Time  tagged  estimates  (state  vector 
and  covariance  matrix)  from  the 
sensor’s  track  stores.  Should  be  in 
the  local  East-North-Up  (ENU) 
reference  frame  (center  at  PRP)  of 
the  sensor. 

ECS  EDR 
Note, 

covariance 
data  not 
available 

SPY 

EPN 

32,33,34 

(This  will 

not  be 

recrded 

during 

normal 

JCIET 

missions 

Yes,  at 
the 

terminal 
on  the 

NCS  bus 

Navigational  data 

N/A 

SPY 

EPN  108 

N/A 

Same  as 
above 

WGS-84  position  of  Platform 
Reference  Point  (PRP). 

SPY 

EPN  108 

Yes  in 
TPS-59 
SGF  files, 
ADCP  - 
Pass  100, 
Internal 
ADCP 
ddb 

Same  as 
above 

Rate  of  change  of  position  of  PRP 

N/A 

SPY 

EPN 

N/A 

Same  as 
above 

108 

Navigational  data  (a  subset  of  the 
data  listed  below) 
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Orientation  of  PRP  (only  if  state 
vector  and  covariance  matrix  are  not 
in  the  local  ENU  reference  frame  of 
sensor). 

ECS  EDR 
note  (3) 

Matrices 
are  in 

ENU 

TPS-59 
SGF  files 

Same  as 
above 

Rate  of  change  of  orientation  of  PRP 
(only  if  state  vector  and  covariance 
matrix  are  not  in  the  local  ENU 
reference  frame  of  sensor). 

N/A 

Matrices 
are  in 

ENU 

TPS-59 
SGF  files 

Same  as 
above . 

3 

Time  tagged  positional  data  from  the 
around  truth  aircraft 

N/A 

N/A 

SGF  files 

Same  as 
above 

r 

Uncertainty  in  accuracy  of  positional 
data  form  the  ground  truth  aircraft 

N/A 

N/A 

SGF  files 

Same  as 
above 

Table  1 0.  Data  Registration 


Data  Registration 

PATRIOT 

AEGIS 

E-2C 

TAOM  V4 

AWACS 

r 

Time  tagged  positional  data  from  the 
around  truth  aircraft 

N/A 

N/A 

N/A 

N/A 

Uncertainty  in  accuracy  of  positional 
data  form  the  ground  truth  aircraft 

N/A 

N/A 

N/A 

N/A 

r 

Time  tagged  measurement  data  on 
the  position  of  the  ground  truth  aircraft 
from  sensor 

EDR, 
sina.sinb, 
range  and 
xvz 

SPY  War 
Diary 

AN/TPS- 
59  SGF 
files 

Yes, 

AOCP 

r 

Uncertainty  of  sensor  measurement 
(if  available) 

N/A 

N/A 

SGF 

bias? 

Yes,RSIP 

5 

Navigational  data  of  the  Platform 
Reference  Point  (PRP)  see  note  1 

Yes,  at 
the 

terminal 
on  the 

NCS  bus 

WGS-84  position*  of  Platform 
Reference  Point  (PRP). 

EDR, 
Lat.Long, 
Alt  from 
GPS  on 
Antenna 
(3) 

C&D  War 
Diary, 

EPN 

43,92,195 
,198.  C2P 
batch 
sets,  SGS 

TPS-59, 

TAOM, 

ADCP 

Same  as 
above 

WGS-84  velocity  of  PRP,  i.e.local 
level  ENU  velocity  vector 

N/A 

SPY  EPN 
108 

N/A 

Same  as 
above 

Attitude  vector  of  PRP  orientation 
relative  to  (ENU)  local  level,  i.e.  roll , 
pitch,  and  heading  data 

EDR,RPY 
computed 
by  NFS 
system 
(3) 

SPY  EPN 
108 

N/A 

Same  as 
above 

Rate  of  change  of  attitude  vector  of 
PRP,  i.e.  roll  rate,  pitch  rate,  and 
headinq  rate 

N/A 

SPY  EPN 
108 

N/A 

Same  as 
above 

Lever  arms  from  PRP  to  center  of 
sensor  aperture  and  leverarm 
between  navigation  system  and  PRP 
(if  non-zero) 

N/A 

9 

N/A 
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Surveyed  orientation  of  sensor  frame 
with  respect  to  local  level  body 
coordinates  of  the  platform  reference 
frame  of  PRP 

N/A 

N/A 

? 

???  is  this 
the  same 
as  #3 

" 

*WGS-84  Altitude  is  with  respect  to 
Mean-Sea-Level  (MSL) 

Note  1 :  The  PRP  may  be  coincident  wit 
primary  navigation  system  is  defined  as 
navigation  systems,  then  data  should  be 
being  used  for  track  determination 

h  the  origin  o 
the  one  usee 
j  recorded  fo 

the  primary  navigation  system,  where  the 
for  track  generation.  If  a  platform  has  dual 
both,  along  with  an  indication  of  which  one  is 

(3)  Data  collection  must  be  toggled  off/on  to  collect  this  information  1 

Table  1 1 .  Correlation/Decorrelation 


Automatic  Local-to-Remote 
Correlation/Decorrelation 

PATRIOT 

AEGIS 

E-2C 

TAOM  V4 

AWACS 

I 

All  tracks  from  Link  16  -  tracks  held 
by  sensors:  there  should  be  only  one 
copy  for  each  target,  which  should  be 
the  sensor  with  R2.  For  each  track, 
we  will  need  to  determine  if  there  are 
two  or  more  tracks  that  are  actually 
the  same  by  comparing  tracks  that 
are  in  the  vicinity  of  each  other 
(duals). 

Yes,  ICC 
EDR  and 
CEES  link 
buffers 

C&D  War 
Diary, 

C2P 

Batch 
sets,  SGS 

TADIL-J 

Messages 

(TAOM 

ddb 

format). 

All  Link  16 
messages 
into  and 
out  of  the 
ADCP  are 
being 
recorded. 
TAOM 
ddb 
format, 
Global 
Track  File 
(TAOM 
ddb 

format). 
ADCP 
Internal 
ddb,  Pass 
100,  JITs 

Yes 

AOCP 

Time  tagged  WGS-84  positional  data* 
from  the  ground  truth  aircraft  provide 
the  best  source  for  these  track 
comparisons. 

N/A 

N/A 

N/A 

???  N/A 

*  WGS-84  altitude  is  with  respect  to 
Mean-Sea-Level.  ! 
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Table  12.  ID  Processing 


— 

ID  Processing 

PATRIOT 

AEGIS 

E-2C 

TAOM  V4 

AWACS 

r 

IFF/SI  F  interrogation  requests  from 
combat  system 

ECS  EDR 

C&D  War 
Diary  EPN 
75,76,  Link 
57,100,101, 
248,265,568 
,569  (IFF) 

35,  53 

(Operator 

alerts  and 

actions)C2P 

Batch 

sets.SGS 

Yes 

AOCP 

r 

IFF/SIF  interrogation  response  from 
transponder  (should  include  all 
candidates) 

ECS  EDR 

Same  as 
above 

Same  as 
above 

r 

Central  track  stores  IFF/SIF  data 

ICC  EDR 

Same  as 
above 

Same  as 
above 

Table  1 3.  Formation  T rack  Assessment 


— 

Formation  Track  Assessment 

PATRIOT 

AEGIS 

E-2C 

TAOM  V4 

AWACS 

T 

Central  T rack  Stores  T rack 

Information 

ICC  EDR 

Yes.  In  the 

TAOM  - 

JITS.trd, 

Pass  100, 

Internal 

TAOM 

ddb 

Yes 

AOCP 

~2 

Data  Link  Input/Output  Buffers 

CEES 

Yes  in 
ADCP. 
JITS.trd 
files, 

PASS 

100, 

Internal 

ADCP 

ddb 

~3 

T ruth  T rack  information  on  all 
possible  A/C 

N/A 

T 

PATRIOT  Formation  Association  and 
Assessment  Decision  Aid  (FAADA)  * 

TACO 

recording 

*only  applicable  to  PATRIOT 

Page  24 
Version  1.0A.1.0 
4  April  2002 


Table  14.  Model  and  Simulation/Stimulation  Fidelity 


Model  and 

Simulation/Stimulation  Fidelity 

PATRIOT 

AEGIS 

E-2C 

TAOM  V4 

AWACS 

1 

Sensor  configuration 
(detection/tracking  sectors,  sector 
blankinq,  Track  Production  Areas) 

ECS  EDR 

2 

Automatic  ID  processor 
configuration  (IID  and/or 
IFF/SlFdoctrine  areas,  types  etc...) 

3 

Data  link  configuration 
(transmit/receive  filters,  Dial-a-TQ 
settings,  etc...) 

Table  15.  PPLI  Accuracy 


— 

PPLI  Accuracy 

PATRIOT 

AEGIS 

E-2C 

TAOM  V4 

AWACS 

FIGHTERS 

r 

Time  tagged  ground  truth  WGS-84 
positional  data  for  platform  PPLI 
(see  note  1) 

N/A 

N/A 

No 

2~ 

Uncertainty  in  accuracy  of  ground 
truth  WGS-84positional  data  *  for 
platform  PPLI  (if  available) 

N/A 

N/A 

No 

r 

PPLIs  from  the  platforms  of  interest 
from  Link  16  received  messages, 
and/or  Link  16  navigation  Terminal 
Input  Messages  (TIMs)  and 

Terminal  Output  Messages  (TOMs). 
(See  note  2) 

CEES  (3) 

C2P 

Batch 

sets 

Yes 

TOM 

r 

Uncertainty  in  the  accuracy  of  the 

PPLIs  from  Link  16  received 
messages,  and/or  Link  1 6  navigation 
TIMS  and  TOMs. 

N/A 

C2P  J2.2i 
message 

Yes 

TOM 

*WGS-84  altitude  is  with  respect  to 
Mean-Sea-Level  (MSL) 

Note  1:  Stationary  Link  16  Ground 
Terminals  must  supply  surveyed 
WGS-84  position  of  the  Link  16 
transmit  antenna.  A  new  survey  must 
be  made  for  any  change  in  antenna 
location.  Also,  the  cable  length,  or 
cable  delay  between  the  Link  16 
Terminal  and  the  transmit  antenna 
must  be  provided 

Note  2:  PPLIs  for  the  platform  being 
evaluated  may  be  recorded  by  any 
Terminal  within  line-of-sight 
connectivity  of  the  platform.  The 
platform  Terminal  TIMS  and  TOMS 
must  be  recorded  at  the  platform  via 
MUX  tape  recording  or  Terminal 
Support  Port  Tape  Recording 
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(3)  CEES  is  the  only  available  data 
collection _ 


Table  1 6.  Multi-Link  T ranslation  Forwarding 


— 

Multi-Link  Translation  Forwarding 

PATRIOT 

AEGIS 

E-2C 

TAOM  V4 

AWACS 

r 

Data  link  input/output  buffers 

CEES 

C&D  War 
Diary, 

C2P 

Batch 

Sets 

Yes. 

JITS.trd 

files 

(TAOM 

and 

ADCP), 

PASS 

100, 

Internal 

ADCP 

ddb 

N/A 

5~ 

Central  Track  Stores  Information 

ICC  EDR 

Yes. In  the 

TAOM- 

JITS.trd, 

Pass  100, 

Internal 

TAOM 

ddb 

4.1.2  SIAP  Metrics 

The  SAT  will  examine  the  available  data  and  compute  the  SIAP  attributes  using 
the  PET  tool.  After  reviewing  the  results,  root-cause  analysis  will  begin  for  diagnosing 
the  problems  that  occurred  during  the  event. 

4.1.3  Root-Cause  Analysis 

To  properly  address  root-cause  analysis  issues,  data  in  addition  to  the 
participants’  PET-output  file  will  be  required.  Participants  are  to  provide  the  SAT  with 
contact  information  for  individuals  who  can  contribute  to  the  root-cause  analysis  effort 
as  shown  in  Table  17. 
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Table  17.  Root-Cause  Analysis  Participant  Points-of-Contact  Worksheet 


Name  of  Person/Organization  providing  analysis 

Erik  Van  Fleet/  NSWC  Corona 

System  Identification  (incl.  s/w  version) 

SIAP  SE  Analysis 

System  issues  analysis  (including  TTPs) 

Name  of  Person/Organization  providing  analysis 

Phil  Baker/  NSWC  Corona 

System  Identification  (incl.  s/w  version) 

Data  Link  Analysis/WAM  operator 

System  issues  analysis  (including  1  I  Ps) 

Name  of  Person/Organization  providing  analysis 

Tony  Pham/  NSWC  Corona 

System  Identification  (incl.  s/w  version) 

AEGIS  C&D  analysis 

System  issues  analysis  (including  TTPs) 

Name  of  Person/Organization  providing  analysis 

Tom  Potochny/  NSWC  Corona 

System  Identification  (incl.  s/w  version) 

CEP  Analysis 

System  issues  analysis  (including  1  1  Ps) 

Name  of  Person/Organization  providing  analysis 

Greg  Hemmila/  NSWC  Corona 

Patty  Pfouts/  NSWC  Corona 

Ronald  Meiia/  NSWC  Corona 

System  Identification  (incl.  s/w  version) 

PET  Trackmatching 

JCIET  Data  base/TORs 

SIAP  SE  IT  assistance 

System  issues  analysis  (including  TTPs) 

4.1. 3.1  Events  of  Interest 

Events  of  Interest  (EOls)  such  as  a  leaker  or  a  fratricide  will  be  identified  on-site 
at  the  event  as  well  as  during  the  root-cause  phase  of  the  analysis  after  the  event. 

4.1. 3.2  Test  Observable  Reports 

The  SAT  will  generate,  collect  and  attempt  to  resolve  Test  Observation  Reports 
(TORs)  during  the  on-site  analysis  effort.  Any  TOR  that  is  unresolved  at  the  end  of  the 
on-site  analysis  effort  will  be  considered  for  further  analysis  during  the  post-event 
analysis  effort. 
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4.1.4  Perturbation  Analysis 


The  SAT  will  conduct  perturbation  analysis  for  the  JCIET  02  event  in  accordance 
with  the  Standard  DMAP. 


5.  REPORTING 

5.1  Quick-Look  Report 

A  quick-look  report  will  be  generated  by  the  SAT  to  summarize  the  initial  results 
of  the  JCIET  02  event.  The  report  will  include  preliminary  PET-output  results. 

5.2  Technical  Report 

The  SAT  will  generate  and  provide  a  final  Technical  Report  (TR)  for  the  JCIET  02 
event.  The  report  will  include  a  summary  of  the  IADS  performance  as  measured  by  the 
SIAP  attributes  and  associated  root-cause  analysis.  The  test  report  will  follow  the 
formatting  recommended  in  the  standard  DMAP. 

5.3  Lessons  Learned 

Lessons  learned  from  the  JCIET’02  event,  including  issues  with  the  location, 
logistics,  planning,  execution,  and  analysis  will  be  generated  by  the  SAT  with  inputs 
from  the  participants. 

5.4  Unresolved  Issues 

Any  unresolved  issues  will  be  documented  and  included  in  the  appropriate 
reports. 
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